## OVERVIEW

The code in this replication package conducts the analysis for the manscript 
"Understanding the Success and Failure of Online Political Debate: Experimental
Evidence using Large Language Models". All code is in R. Two .R files run the
code necessary to generate all figures and tables in the main manuscript and
appendix. No file will take a substantial amount of time to run.


## DATA AVAILABILITY

All data necessary for all statistical tables, statistics-based figures, and
summary and inferenetial statistics that are in the manuscript and appendix are
provided. Figure S1, which is a diagram, is not generated by data in this
archive.


## FILES

  ./2023 006 Rasmussen IFS Ethics approval.pdf
  Ethics approval from the University of Copenhagen for the experiment.

  ./Figures/
  Files in this directory are those that are used in the main manuscript and in
  the appendix. One can thus compare these to those outputted from the
  replication script.


## DATASET LIST

The following data are provided in the replication archive:

  ./Data/Data.rds
  Survey experimental data that allow for all main analysis in the manscript.

  ./Data/anes_timeseries_2020_stata_20220210.dta
  American National Election Survey data (2020) for which to construct balance
  tables comparing Democratic and Republican voters in the experiment.

  ./Data/bes2019.dta
  British Election Survey data (2019) for which to construct balance
  tables comparing Labour and Conservative voters in the experiment.


## DESCRIPTION OF PROGRAMS/CODE

  ./ggplot_theme.R
  A ggplot2 theme that is loaded from each of the analysis R files below.

  
  # MAIN ANALYSIS

  The following two files conduct all of the analysis for the article from the
  data that are found in ./Data/. All figures and tables are labeled at the
  relevant part of the the code (e.g. "TABLE S11", "FIGURE 3") so that
  replicators can easily search for any specific item to replicate in the two
  R files below. These files can be run in any order.

  ./1_Analysis.R
  Computes all Figures, Tables, and statistics regarding the experiment aside
  from statistics for validating responses from the LLM
  - Figures 1-6 and Figures S2-S31
    - Figure S1 is not included in the analysis file because it is a diagram
  - Tables S1-S19

  ./2_Validation.R
  Computes summary statistics for the validation task.
  - i.e. information found in the Supplementary Material sub-section
         "LLM response consistency and validation"


## R LIBRARIES

  R (4.4.2)
    - modelsummary:    2.3.0
    - broom:           1.0.7
    - haven:           2.5.4
    - xtable:          1.8.4
    - stringr:         1.5.1
    - cowplot:         1.1.3
    - colorspace:      2.1.1
    - marginaleffects: 0.25.0
    - icr:             0.6.6
    - ltm:             1.2.0
    - tidyverse:       2.0.0
    - dplyr:           1.1.4


